1. Field of the Invention
The present invention relates to an image pickup apparatus having a flicker correction function for suppressing occurrence of flicker in an image signal captured by an XY address image pickup element such as a complementary metal-oxide semiconductor (CMOS) sensor, and to a control method for the image pickup apparatus.
2. Description of the Related Art
Image signals captured using existing image pickup apparatuses including image pickup elements such as charge-coupled device (CCD) sensors or CMOS sensors may include flicker. No flicker occurs when images are captured in light source environments where the amount of light emitted from a light source is constant and unchanged, such as sunlight.
Flicker occurs, for example, in a case where a light source having a light emission characteristic (brightness) that changes at a constant frequency, such as a fluorescent lamp, is used and in a case where a cycle in which the brightness of the light source changes is not synchronized with a sampling frequency of an electronic shutter of an image pickup element (a timing of an accumulation period).
Flicker is a phenomenon in which due to a deviation between the timing of changes in brightness of a light source in a photographic environment and the timing of electric charge accumulation of an image pickup element, periodic differences in brightness occur on a generated output image. There are two types of flicker depending on the electric charge accumulation method of the image pickup element.
In CCD sensors, a global shutter system in which the timing of electric charge accumulation is matched in units of surfaces of image pickup elements is employed. Thus, the brightness changes field by field, i.e., in units of generated output images (referred to as field flicker).
FIG. 4 shows occurrence of flicker in an image pickup element employing the global shutter system, such as a CCD sensor.
As shown in FIG. 4, in a case where an image is captured in an environment where the brightness changes at a constant frequency, since the integrated electric charge value generated by an amount of light emitted from a light source is different from one field to another according to an accumulation period, a brightness of an output image changes field by field.
A specific example will now be described.
In a non-inverter-type fluorescent lamp that uses a power supply frequency of 50 Hz, a change in brightness occurs at intervals of 100 Hz (intervals twice the power supply frequency).
In such a light source environment, if an image is captured using the NTSC (National Television System Committee) system (60 fields per second), integrated electric charge values generated by an amount of light emitted from the light source, which are accumulated for individual fields, are not equal to each other. Thus, a brightness of an output image changes every field.
The reason is as follows. With the use of a power supply frequency of 50 Hz, 100 peaks as shown in FIG. 4 are produced per second, and 60 accumulation timings are defined with respect to the 100 peaks. If the peaks are equally divided into 60 sections, starting from the first peak, the integrated electric charge value differs depending on the peak.
In order to suppress such field flicker generated in an image pickup element such as a CCD sensor, an accumulation period is fixedly set to 1/100 seconds. The timing of changes in brightness of a light source is synchronized with the timing of the electric charge accumulation of the image pickup element to suppress flicker.
With this method, the integrated electric charge values generated by accumulation of an amount of light emitted from the light source can be made constant regardless of the timing of exposure, and the influence of flicker can completely be suppressed.
Recently, CMOS sensors with lower power consumption and lower production cost than CCD sensors have attracted attention.
XY address image pickup elements such as CMOS sensors employ a rolling shutter system in which the timing of electric charge accumulation is matched with each of the accumulation lines of image pickup elements. The occurrence of flicker would not be suppressed by the technique described above.
FIGS. 5A and 5B show occurrence of flicker in an image pickup element employing the rolling shutter system, such as a CMOS sensor.
As shown in FIG. 5A, the integrated electric charge value generated by an amount of light emitted from a light source is different from one accumulation line to another (for example, accumulation line 2 has a larger integrated value than accumulation line 1). As a result, unlike an image pickup element such as a CCD sensor, flicker appears in an output image within a field. As shown in FIG. 5B, high-brightness (bright) and low-brightness (dark) areas are produced, like a periodic wave, within the output image (referred to as line flicker).
In an image pickup element employing the rolling shutter system, such as a CMOS sensor, in order to suppress line flicker, a gain of each accumulation line of an image pickup element is adjusted using a gain correction circuit so that a difference in brightness due to a detected flicker component can be canceled.
In this technique, the detected flicker component is approximated to a sine wave, and the gain is adjusted so as to cancel the sine wave to perform correction.
FIG. 6 is a diagram showing gain adjustment in a flicker correction process of the related art.
As shown in FIG. 6, a negative gain is applied to an accumulation line having a higher value than an average brightness value of an entire screen, and a positive gain is applied to an accumulation line having a lower value than the average brightness value to thereby adjust the brightness of the overall output image.
In this method, the flicker component that has occurred in the output image can be corrected by gain adjustment.
However, in a case where a gain value has been amplified because of a dark object to be photographed, a problem occurs. In this case, the original gain value is further amplified to a large value by gain adjustment. Thus, the noise component is also amplified in an accumulation line for which, as flicker correction, the gain is amplified to the positive side.
As a result, noise is enhanced and the signal-to-noise (S/N) ratio is reduced.
Furthermore, in an image subjected to flicker correction, gain values differ for individual accumulation lines of the image pickup element. The levels of the noise component also differ for the individual accumulation lines accordingly.
In addition, the difference between a gain value of an accumulation line for which gain adjustment is performed to the positive side and a gain value of an accumulation line for which gain adjustment is performed to the negative side is large. Thus, as with changes in brightness in line flicker, a noise component is generated like a periodic wave.
In gain adjustment, a noise component is amplified using an exponential function. If the difference between two gain values is large, a wave generated by the noise component appears more noticeable.
Japanese Patent Laid-Open No. 2006-303815 discloses a flicker correction method. In the flicker correction method, a flicker correction signal is generated from an output image before and after flicker correction, and the flicker correction signal is subtracted from the output image before flicker correction to correct flicker.
Then, a noise component is removed using a low-pass filter to improve the S/N ratio.
In the method disclosed in Japanese Patent Laid-Open No. 2006-303815, the low-pass filter can reduce the noise component but can also remove a high-frequency component, resulting in a reduction in the resolution of an output image.
It is therefore difficult to adjust the difference in brightness between accumulation lines in an output image, which is caused by flicker, while suppressing an increase in noise components and maintaining a high resolution.